Security News
Research
Data Theft Repackaged: A Case Study in Malicious Wrapper Packages on npm
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
The 'tldts' npm package is a powerful tool for parsing and manipulating domain names. It helps in extracting various parts of a domain, such as the top-level domain (TLD), subdomain, and domain name. It is particularly useful for tasks involving URL validation, domain categorization, and security checks.
Extract Domain Parts
This feature allows you to parse a URL and extract its components such as the subdomain, domain, and TLD. The code sample demonstrates how to parse a URL and log the parsed components.
const tldts = require('tldts');
const parsed = tldts.parse('https://sub.example.co.uk/path');
console.log(parsed);
Get Domain Without Subdomain
This feature extracts the domain name without the subdomain. The code sample shows how to get the domain name from a URL.
const tldts = require('tldts');
const domain = tldts.getDomain('https://sub.example.co.uk/path');
console.log(domain);
Get Public Suffix
This feature retrieves the public suffix (TLD) of a given URL. The code sample demonstrates how to extract the public suffix from a URL.
const tldts = require('tldts');
const publicSuffix = tldts.getPublicSuffix('https://sub.example.co.uk/path');
console.log(publicSuffix);
Is Valid URL
This feature checks if a given URL is valid. The code sample shows how to validate a URL.
const tldts = require('tldts');
const isValid = tldts.isValid('https://sub.example.co.uk/path');
console.log(isValid);
The 'psl' package is a similar tool that provides functions to parse domain names and extract the public suffix. It is often used for similar purposes as 'tldts', such as URL validation and domain categorization. However, 'tldts' offers more comprehensive parsing capabilities and additional features like subdomain extraction.
The 'url-parse' package is a robust URL parser that can decompose URLs into their constituent parts. While it provides general URL parsing capabilities, it does not specialize in domain-specific parsing like 'tldts'. 'tldts' offers more focused functionality for domain and TLD extraction.
The 'parse-domain' package is another tool for parsing domain names and extracting subdomains, domains, and TLDs. It is similar to 'tldts' but may not be as actively maintained or feature-rich. 'tldts' provides a more modern and comprehensive solution for domain parsing.
tldts
is a JavaScript library to extract hostnames, domains, public suffixes, top-level domains and subdomains from URLs.
Features:
umd
, esm
, cjs
bundles and type definitionsnpm install --save tldts
Using the command-line interface:
$ npx tldts 'http://www.writethedocs.org/conf/eu/2017/'
{
"domain": "writethedocs.org",
"domainWithoutSuffix": "writethedocs",
"hostname": "www.writethedocs.org",
"isIcann": true,
"isIp": false,
"isPrivate": false,
"publicSuffix": "org",
"subdomain": "www"
}
Programmatically:
const { parse } = require('tldts');
// Retrieving hostname related informations of a given URL
parse('http://www.writethedocs.org/conf/eu/2017/');
// { domain: 'writethedocs.org',
// domainWithoutSuffix: 'writethedocs',
// hostname: 'www.writethedocs.org',
// isIcann: true,
// isIp: false,
// isPrivate: false,
// publicSuffix: 'org',
// subdomain: 'www' }
Modern ES6 modules import is also supported:
import { parse } from 'tldts';
Alternatively, you can try it directly in your browser here: https://npm.runkit.com/tldts
tldts.parse(url | hostname, options)
tldts.getHostname(url | hostname, options)
tldts.getDomain(url | hostname, options)
tldts.getPublicSuffix(url | hostname, options)
tldts.getSubdomain(url, | hostname, options)
tldts.getDomainWithoutSuffix(url | hostname, options)
The behavior of tldts
can be customized using an options
argument for all
the functions exposed as part of the public API. This is useful to both change
the behavior of the library as well as fine-tune the performance depending on
your inputs.
{
// Use suffixes from ICANN section (default: true)
allowIcannDomains: boolean;
// Use suffixes from Private section (default: false)
allowPrivateDomains: boolean;
// Extract and validate hostname (default: true)
// When set to `false`, inputs will be considered valid hostnames.
extractHostname: boolean;
// Validate hostnames after parsing (default: true)
// If a hostname is not valid, not further processing is performed. When set
// to `false`, inputs to the library will be considered valid and parsing will
// proceed regardless.
validateHostname: boolean;
// Perform IP address detection (default: true).
detectIp: boolean;
// Assume that both URLs and hostnames can be given as input (default: true)
// If set to `false` we assume only URLs will be given as input, which
// speed-ups processing.
mixedInputs: boolean;
// Specifies extra valid suffixes (default: null)
validHosts: string[] | null;
}
The parse
method returns handy properties about a URL or a hostname.
const tldts = require('tldts');
tldts.parse('https://spark-public.s3.amazonaws.com/dataanalysis/loansData.csv');
// { domain: 'amazonaws.com',
// domainWithoutSuffix: 'amazonaws',
// hostname: 'spark-public.s3.amazonaws.com',
// isIcann: true,
// isIp: false,
// isPrivate: false,
// publicSuffix: 'com',
// subdomain: 'spark-public.s3' }
tldts.parse(
'https://spark-public.s3.amazonaws.com/dataanalysis/loansData.csv',
{ allowPrivateDomains: true },
);
// { domain: 'spark-public.s3.amazonaws.com',
// domainWithoutSuffix: 'spark-public',
// hostname: 'spark-public.s3.amazonaws.com',
// isIcann: false,
// isIp: false,
// isPrivate: true,
// publicSuffix: 's3.amazonaws.com',
// subdomain: '' }
tldts.parse('gopher://domain.unknown/');
// { domain: 'domain.unknown',
// domainWithoutSuffix: 'domain',
// hostname: 'domain.unknown',
// isIcann: false,
// isIp: false,
// isPrivate: true,
// publicSuffix: 'unknown',
// subdomain: '' }
tldts.parse('https://192.168.0.0'); // IPv4
// { domain: null,
// domainWithoutSuffix: null,
// hostname: '192.168.0.0',
// isIcann: null,
// isIp: true,
// isPrivate: null,
// publicSuffix: null,
// subdomain: null }
tldts.parse('https://[::1]'); // IPv6
// { domain: null,
// domainWithoutSuffix: null,
// hostname: '::1',
// isIcann: null,
// isIp: true,
// isPrivate: null,
// publicSuffix: null,
// subdomain: null }
tldts.parse('tldts@emailprovider.co.uk'); // email
// { domain: 'emailprovider.co.uk',
// domainWithoutSuffix: 'emailprovider',
// hostname: 'emailprovider.co.uk',
// isIcann: true,
// isIp: false,
// isPrivate: false,
// publicSuffix: 'co.uk',
// subdomain: '' }
Property Name | Type | Description |
---|---|---|
hostname | str | hostname of the input extracted automatically |
domain | str | Domain (tld + sld) |
domainWithoutSuffix | str | Domain without public suffix |
subdomain | str | Sub domain (what comes after domain ) |
publicSuffix | str | Public Suffix (tld) of hostname |
isIcann | bool | Does TLD come from ICANN part of the list |
isPrivate | bool | Does TLD come from Private part of the list |
isIP | bool | Is hostname an IP address? |
These methods are shorthands if you want to retrieve only a single value (and
will perform better than parse
because less work will be needed).
Returns the hostname from a given string.
const { getHostname } = require('tldts');
getHostname('google.com'); // returns `google.com`
getHostname('fr.google.com'); // returns `fr.google.com`
getHostname('fr.google.google'); // returns `fr.google.google`
getHostname('foo.google.co.uk'); // returns `foo.google.co.uk`
getHostname('t.co'); // returns `t.co`
getHostname('fr.t.co'); // returns `fr.t.co`
getHostname(
'https://user:password@example.co.uk:8080/some/path?and&query#hash',
); // returns `example.co.uk`
Returns the fully qualified domain from a given string.
const { getDomain } = require('tldts');
getDomain('google.com'); // returns `google.com`
getDomain('fr.google.com'); // returns `google.com`
getDomain('fr.google.google'); // returns `google.google`
getDomain('foo.google.co.uk'); // returns `google.co.uk`
getDomain('t.co'); // returns `t.co`
getDomain('fr.t.co'); // returns `t.co`
getDomain('https://user:password@example.co.uk:8080/some/path?and&query#hash'); // returns `example.co.uk`
Returns the domain (as returned by getDomain(...)
) without the public suffix part.
const { getDomainWithoutSuffix } = require('tldts');
getDomainWithoutSuffix('google.com'); // returns `google`
getDomainWithoutSuffix('fr.google.com'); // returns `google`
getDomainWithoutSuffix('fr.google.google'); // returns `google`
getDomainWithoutSuffix('foo.google.co.uk'); // returns `google`
getDomainWithoutSuffix('t.co'); // returns `t`
getDomainWithoutSuffix('fr.t.co'); // returns `t`
getDomainWithoutSuffix(
'https://user:password@example.co.uk:8080/some/path?and&query#hash',
); // returns `example`
Returns the complete subdomain for a given string.
const { getSubdomain } = require('tldts');
getSubdomain('google.com'); // returns ``
getSubdomain('fr.google.com'); // returns `fr`
getSubdomain('google.co.uk'); // returns ``
getSubdomain('foo.google.co.uk'); // returns `foo`
getSubdomain('moar.foo.google.co.uk'); // returns `moar.foo`
getSubdomain('t.co'); // returns ``
getSubdomain('fr.t.co'); // returns `fr`
getSubdomain(
'https://user:password@secure.example.co.uk:443/some/path?and&query#hash',
); // returns `secure`
Returns the public suffix for a given string.
const { getPublicSuffix } = require('tldts');
getPublicSuffix('google.com'); // returns `com`
getPublicSuffix('fr.google.com'); // returns `com`
getPublicSuffix('google.co.uk'); // returns `co.uk`
getPublicSuffix('s3.amazonaws.com'); // returns `com`
getPublicSuffix('s3.amazonaws.com', { allowPrivateDomains: true }); // returns `s3.amazonaws.com`
getPublicSuffix('tld.is.unknown'); // returns `unknown`
localhost
and custom hostnamestldts
methods getDomain
and getSubdomain
are designed to work only with known and valid TLDs.
This way, you can trust what a domain is.
localhost
is a valid hostname but not a TLD. You can pass additional options to each method exposed by tldts
:
const tldts = require('tldts');
tldts.getDomain('localhost'); // returns null
tldts.getSubdomain('vhost.localhost'); // returns null
tldts.getDomain('localhost', { validHosts: ['localhost'] }); // returns 'localhost'
tldts.getSubdomain('vhost.localhost', { validHosts: ['localhost'] }); // returns 'vhost'
tldts
made the opinionated choice of shipping with a list of suffixes directly
in its bundle. There is currently no mechanism to update the lists yourself, but
we make sure that the version shipped is always up-to-date.
If you keep tldts
updated, the lists should be up-to-date as well!
tldts
is the fastest JavaScript library available for parsing hostnames. It is able to parse millions of inputs per second (typically 2-3M depending on your hardware and inputs). It also offers granular options to fine-tune the behavior and performance of the library depending on the kind of inputs you are dealing with (e.g.: if you know you only manipulate valid hostnames you can disable the hostname extraction step with { extractHostname: false }
).
Please see this detailed comparison with other available libraries.
tldts
is based upon the excellent tld.js
library and would not exist without
the many contributors who worked on the project:
This project would not be possible without the amazing Mozilla's public suffix list. Thank you for your hard work!
FAQs
Library to work against complex domain names, subdomains and URIs.
The npm package tldts receives a total of 3,723,455 weekly downloads. As such, tldts popularity was classified as popular.
We found that tldts demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.